Data extraction methods for systematic review (semi)automation: Update of a living systematic review [version 2; peer review: 3 approved]

Background: The reliable and usable (semi)automation of data extraction can support the field of systematic review by reducing the workload required to gather information about the conduct and results of the included studies. This living systematic review examines published approaches for data extraction from reports of clinical studies. Methods: We systematically and continually search PubMed, ACL Anthology, arXiv, OpenAlex via EPPI-Reviewer, and the dblp computer science bibliography. Full text screening and data extraction are conducted within an open-source living systematic review application created for the purpose of this review. This living review update includes publications up to December 2022 and OpenAlex content up to March 2023. Results: 76 publications are included in this review. Of these, 64 (84%) of the publications addressed extraction of data from abstracts, while 19 (25%) used full texts. A total of 71 (93%) publications developed classifiers for randomised controlled trials. Over 30 entities were extracted, with PICOs (population, intervention, comparator, outcome) being the most frequently extracted. Data are available from 25 (33%), and code from 30 (39%) publications. Six (8%) implemented publicly available tools Conclusions: This living systematic review presents an overview of (semi)automated data-extraction literature of interest to different types of literature review. We identified a broad evidence base of publications describing data extraction for interventional reviews and a small number of publications extracting epidemiological or diagnostic accuracy data. Between review updates, trends for sharing data and code increased strongly: in the base-review, data and code were available for 13 and 19% respectively, these numbers increased to 78 and 87% within the 23 new publications. Compared with the base-review, we observed another research trend, away from straightforward data extraction and towards additionally extracting relations between entities or automatic text summarisation. With this living review we aim to review the literature continually

Assessing Open Science practices in physical activity behaviour change intervention evaluations

This article is a preprint and has not been peer-reviewed [what does this mean?]. It reports new medical research that has yet to be evaluated and so should not be used to guide clinical practice.Copyright 2022 The Authors. Objectives Concerns on the lack of reproducibility and transparency in science have led to a range of research practice reforms, broadly referred to as ‘Open Science’. The extent that physical activity interventions are embedding Open Science practices is currently unknown. In this study, we randomly sampled 100 reports of recent physical activity behaviour change interventions to estimate the prevalence of Open Science practices. Methods One hundred reports of randomised controlled trial physical activity behaviour change interventions published between 2018-2021 were identified. Open Science practices were coded in identified reports, including: study pre-registration, protocol sharing, data-, materials- and analysis scripts-sharing, replication of a previous study, open access publication, funding sources and conflict of interest statements. Coding was performed by two independent researchers, with inter-rater reliability calculated using Krippendorff’s alpha. Results 78% of the 100 reports provided details of study pre-registration and 41% provided evidence of a published protocol. 4% provided accessible open data, 8% provided open materials and 1% provided open analysis scripts. 73% of reports were published as open access and no studies were described as replication attempts. 93% of reports declared their sources of funding and 88% provided conflicts of interest statements. A Krippendorff’s alpha of 0.73 was obtained across all coding. Conclusion Open data, materials, analysis and replication attempts are currently rare in physical activity behaviour change intervention reports, whereas funding source and conflict of interest declarations are common. Future physical activity research should increase the reproducibility of their methods and results by incorporating more Open Science practices.Competing interests and funding statement: The authors declare they have no competing interests

Assessing Open Science practices in physical activity behaviour change intervention evaluations

Data availability statement: Data are available in a public, open access repository. All data from this study are available here: https://osf.io/t5gw4/Copyright © Author(s) (or their employer(s)) 2022. Objectives: Concerns on the lack of reproducibility and transparency in science have led to a range of research practice reforms, broadly referred to as ‘Open Science’. The extent that physical activity interventions are embedding Open Science practices is currently unknown. In this study, we randomly sampled 100 reports of recent physical activity randomised controlled trial behaviour change interventions to estimate the prevalence of Open Science practices. Methods: One hundred reports of randomised controlled trial physical activity behaviour change interventions published between 2018 and 2021 were identified, as used within the Human Behaviour-Change Project. Open Science practices were coded in identified reports, including: study pre-registration, protocol sharing, data, materials and analysis scripts sharing, replication of a previous study, open access publication, funding sources and conflict of interest statements. Coding was performed by two independent researchers, with inter-rater reliability calculated using Krippendorff’s alpha. Results: 78 of the 100 reports provided details of study pre-registration and 41% provided evidence of a published protocol. 4% provided accessible open data, 8% provided open materials and 1% provided open analysis scripts. 73% of reports were published as open access and no studies were described as replication attempts. 93% of reports declared their sources of funding and 88% provided conflicts of interest statements. A Krippendorff’s alpha of 0.73 was obtained across all coding. Conclusion: Open data, materials, analysis and replication attempts are currently rare in physical activity behaviour change intervention reports, whereas funding source and conflict of interest declarations are common. Future physical activity research should increase the reproducibility of their methods and results by incorporating more Open Science practices.The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors

Delivering Behaviour Change Interventions : Development of a Mode of Delivery Ontology

Acknowledgements We would like to express our gratitude to the experts who contributed to the open peer-review stages of this study and to Kirsty Atha for the support in annotating papers. Grant information: This work is supported by Wellcome through a collaborative award to The Human Behaviour-Change Project [201524]. MMM is funded by a Marie-Sklodowska-Curie fellowship [EU H2020 EDGE program grant agreement No. 713567].Peer reviewedPublisher PD

Ontologies relevant to behaviour change interventions: a method for their development.

Background: Behaviour and behaviour change are integral to many aspects of wellbeing and sustainability. However, reporting behaviour change interventions accurately and synthesising evidence about effective interventions is hindered by lacking a shared, scientific terminology to describe intervention characteristics. Ontologies are standardised frameworks that provide controlled vocabularies to help unify and connect scientific fields. To date, there is no published guidance on the specific methods required to develop ontologies relevant to behaviour change. We report the creation and refinement of a method for developing ontologies that make up the Behaviour Change Intervention Ontology (BCIO). Aims: (1) To describe the development method of the BCIO and explain its rationale; (2) To provide guidance on implementing the activities within the development method. Method and results: The method for developing ontologies relevant to behaviour change interventions was constructed by considering principles of good practice in ontology development and identifying key activities required to follow those principles. The method's details were refined through application to developing two ontologies. The resulting ontology development method involved: (1) defining the ontology's scope; (2) identifying key entities; (3) refining the ontology through an iterative process of literature annotation, discussion and revision; (4) expert stakeholder review; (5) testing inter-rater reliability; (6) specifying relationships between entities, and; (7) disseminating and maintaining the ontology. Guidance is provided for conducting relevant activities for each step.  Conclusions: We have developed a detailed method for creating ontologies relevant to behaviour change interventions, together with practical guidance for each step, reflecting principles of good practice in ontology development. The most novel aspects of the method are the use of formal mechanisms for literature annotation and expert stakeholder review to develop and improve the ontology content. We suggest the mnemonic SELAR3, representing the method's first six steps as Scope, Entities, Literature Annotation, Review, Reliability, Relationships

Development of an Intervention Setting Ontology for behaviour change: Specifying where interventions take place.

Background: Contextual factors such as an intervention's setting are key to understanding how interventions to change behaviour have their effects and patterns of generalisation across contexts. The intervention's setting is not consistently reported in published reports of evaluations. Using ontologies to specify and classify intervention setting characteristics enables clear and reproducible reporting, thus aiding replication, implementation and evidence synthesis. This paper reports the development of a Setting Ontology for behaviour change interventions as part of a Behaviour Change Intervention Ontology, currently being developed in the Wellcome Trust funded Human Behaviour-Change Project. Methods: The Intervention Setting Ontology was developed following methods for ontology development used in the Human Behaviour-Change Project: 1) Defining the ontology's scope, 2) Identifying key entities by reviewing existing classification systems (top-down) and 100 published behaviour change intervention reports (bottom-up), 3) Refining the preliminary ontology by literature annotation of 100 reports, 4) Stakeholder reviewing by 23 behavioural science and public health experts to refine the ontology, 5) Assessing inter-rater reliability of using the ontology by two annotators familiar with the ontology and two annotators unfamiliar with it, 6) Specifying ontological relationships between setting entities and 7) Making the Intervention Setting Ontology machine-readable using Web Ontology Language (OWL) and publishing online. Re sults: The Intervention Setting Ontology consists of 72 entities structured hierarchically with two upper-level classes: Physical setting including Geographic location, Attribute of location (including Area social and economic condition, Population and resource density sub-levels) and Intervention site (including Facility, Transportation and Outdoor environment sub-levels), as well as Social setting. Inter-rater reliability was found to be 0.73 (good) for those familiar with the ontology and 0.61 (acceptable) for those unfamiliar with it. Conclusion: The Intervention Setting Ontology can be used to code information from diverse sources, annotate the setting characteristics of existing intervention evaluation reports and guide future reporting

Data extraction methods for systematic review (semi)automation: Update of a living systematic review [version 2; peer review: 3 approved]

Background: The reliable and usable (semi)automation of data extraction can support the field of systematic review by reducing the workload required to gather information about the conduct and results of the included studies. This living systematic review examines published approaches for data extraction from reports of clinical studies. Methods: We systematically and continually search PubMed, ACL Anthology, arXiv, OpenAlex via EPPI-Reviewer, and the dblp computer science bibliography. Full text screening and data extraction are conducted within an open-source living systematic review application created for the purpose of this review. This living review update includes publications up to December 2022 and OpenAlex content up to March 2023. Results: 76 publications are included in this review. Of these, 64 (84%) of the publications addressed extraction of data from abstracts, while 19 (25%) used full texts. A total of 71 (93%) publications developed classifiers for randomised controlled trials. Over 30 entities were extracted, with PICOs (population, intervention, comparator, outcome) being the most frequently extracted. Data are available from 25 (33%), and code from 30 (39%) publications. Six (8%) implemented publicly available tools Conclusions: This living systematic review presents an overview of (semi)automated data-extraction literature of interest to different types of literature review. We identified a broad evidence base of publications describing data extraction for interventional reviews and a small number of publications extracting epidemiological or diagnostic accuracy data. Between review updates, trends for sharing data and code increased strongly: in the base-review, data and code were available for 13 and 19% respectively, these numbers increased to 78 and 87% within the 23 new publications. Compared with the base-review, we observed another research trend, away from straightforward data extraction and towards additionally extracting relations between entities or automatic text summarisation. With this living review we aim to review the literature continually

Investigating Communication and Social Behaviour Using Wearable Sensor Technology

The behaviour that we exhibit contributes to the message that is communicated to those that we are interacting with and can have an impact on how the message is conveyed and interpreted. Nonverbal behaviour is just as important to be aware of as well as what is being said, as the subtleties of behaviour can impact the outcome of interactions. Advancements in research technologies have allowed us the chance to investigate natural human behaviour is a variety of settings outside of the laboratory, however, some gaps in the understanding of behaviour exist. The aim of this thesis is to investigate communication and social interactions in a variety of settings, paying particular attention to the methods of data collection, specifically the use of wearable sensors, to investigate phenomenon from social psychology. The thesis aims to address three specific research questions; 1) if can we predict stress using a combination of nonverbal behavioural cues along with physiological measurements, 2) understand the factors affecting happiness and productivity in the workplace from features of communication taken from wearable sensors and 3) determine the stressors that can be characterised from communication patterns assessed through Call Detail Records and smartphone sensors. The studies presented here focus on the nonverbal aspects of communication that can be measured through wearable and sensing devices. In the three types of scenarios that are detailed in the different chapters, the interactions considered are face to face meetings in a one on one interaction, co-location within a defined space in an organisation and the communications of a widely dispersed community. The interactions are recorded by wearable devices such as the Affectiva Q sensor, the Sociometric Badge, and smartphones equipped with sensing capabilities in the form of the funf and P-OWL platforms for data recording, among other forms of data collection. Each of the studies included aspects of self reported assessments that were used as a ground truth measurement of affect: these were annotations of stress, self reports of fear of negative evaluation, self perception, positive and negative affect and stress, among others. The goal was to examine how to use digital traces of behavioural expressions to have a greater understanding of these interactions and how the way in which we interact with others has an impact on the individual. The work from this thesis adds to the existing literature on these various issues by addressing the research questions from a novel perspective. The studies found support for each of the research questions and by using a mixed methods approach and digital traces from wearable sensors gained insights into how communication impacted the individual, revealing the important aspects of communication and their effect on stress, productivity and well-being

Data from: Investigating the association between social interactions and personality states dynamics

The recent personality psychology literature has coined the name of personality states to refer to states having the same behavioural, affective and cognitive content (described by adjectives) as the corresponding trait, but for a shorter duration. The variability in personality states may be the reaction to specific characteristics of situations. The aim of our study is to investigate whether specific situational factors, that is, different configurations of face-to-face interactions, are predictors of variability of personality states in a work environment. The obtained results provide evidence that within-person variability in personality is associated with variation in face-to-face interactions. Interestingly, the effects differ by type and level of the personality states: adaptation effects for Agreeableness and Emotional Stability, whereby the personality states of an individual trigger similar states in other people interacting with them and complementarity effects for Openness to Experience, whereby the personality states of an individual trigger opposite states in other people interacting with them. Overall, these findings encourage further research to characterize face-to-face and social interactions in terms of their relevance to personality states